[AI-179] [Automation] feat(AI-179): implement /fiona search — browse raw sources without synthesized answer - #81
[AI-179] [Automation] feat(AI-179): implement /fiona search — browse raw sources without synthesized answer#81roberthunterjr with Copilot wants to merge 26 commits into
Conversation
…nthesized answer - Add searchForSources() to llm-caller.js: non-streaming Perplexity call that extracts search_results (with title/snippet) or falls back to citations (URL-only), returning up to 5 normalized NormalizedSource items. - Create src/agent/search-caller.js: thin abstraction layer that re-exports searchForSources from llm-caller and provides formatSearchResults() and escapeMrkdwn() helpers for Slack mrkdwn formatting. - Update src/listeners/commands/fiona.js: replace the coming-soon stub for /fiona search with a real handleSearch() handler — extracts the query, falls back to help when empty, checks rate limits, calls searchForSources, and responds ephemerally with formatted source snippets. - Update src/listeners/commands/command-handler.js: remove SEARCH_NOT_YET_TEXT, add handleSearchViaSay() for the @-mention / assistant-panel paths, update routeCommandViaSay() to dispatch search to handleSearchViaSay(), and remove the (coming soon) annotation from HELP_TEXT. - Update tests: fiona.test.js, command-handler.test.js, app-mention.test.js, message.test.js, escalation.test.js reflect the new real-search behavior. New test file: tests/agent/search-caller.test.js (38 unit tests). Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
…brittle test - Rename handleComingSoonViaSay parameter from subCommand to keyword - Simplify HELP_TEXT test to use regex instead of exact whitespace match Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
- Move SEARCH_ERROR_TEXT export inline next to its definition in search-caller.js - Add err.message alongside err.name in handleSearchViaSay error log Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
|
|
||
| try { | ||
| const response = await perplexityClient.chat.completions.create({ | ||
| model: PERPLEXITY_API_MODEL, |
There was a problem hiding this comment.
This is using whatever the default model for pplx is. Currently this is set to sonar, which is the synthesis model. We should update this to use the search api to reduce the adjustments to extracting sources and limiting count, this API will handle most of that. Please look up the guidance on implementing the search API and apply here.
There was a problem hiding this comment.
Switched searchForSources to the Perplexity Search API (POST https://api.perplexity.ai/search) via native fetch. The Search API directly returns a structured results array (title, url, snippet) so the search_results/citations extraction and fallback logic has been removed. Count limiting is now delegated to the API via the max_results parameter, with normalizeSources as a secondary cap.
Updated search-caller.test.js to mock globalThis.fetch instead of openai.chat.completions.create, and added a test for HTTP error responses. All 570 tests pass. Commit: feat(AI-179): switch searchForSources to Perplexity Search API (POST /search)
…/search) Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
roberthunterjr
left a comment
There was a problem hiding this comment.
@copilot. Looking good overall. A couple of comments from before to address. Also, here is an image of the reponse seen.
| } | ||
|
|
||
| // ─── Source Search ───────────────────────────────────────────────────────── | ||
| const SEARCH_MAX_SOURCES = 5; |
There was a problem hiding this comment.
This was specified in ticket to be 5. Lets make this an env value and set a max gate here to 10, that sends the minimum of the SEARCH_MAX_SOURCES value and 10 when performing the actual API call
There was a problem hiding this comment.
SEARCH_MAX_SOURCES now reads from process.env.SEARCH_MAX_SOURCES (default 5). A hard cap of SEARCH_ABSOLUTE_MAX = 10 is enforced via Math.min(maxSources, 10) before the Perplexity API call — so max_results never exceeds 10 regardless of the env value. Commit: feat(AI-179): env-backed SEARCH_MAX_SOURCES with hard cap
| * @param {Array<import('./llm-caller.js').NormalizedSource>} sources - Normalized source list | ||
| * @returns {string} Slack mrkdwn-formatted search results string | ||
| */ | ||
| export function formatSearchResults(query, sources) { |
There was a problem hiding this comment.
This is looking pretty good. One issue I see is the response items appear as a long, unformatted list, making it difficult to read. Lets improve the readabilitiy, perhaps formatting the snippet and truncating the content more so the total number of sources is easier to see.
There was a problem hiding this comment.
Added a truncateSnippet helper in search-caller.js that:
- Collapses newlines and extra whitespace to a single space
- Strips
**bold**markers (using[\s\S]*?to handle asterisks inside bold text) and### headingmarkers - Truncates to 150 chars at a word boundary and appends
…
The SNIPPET_MAX_CHARS limit is also env-backed via SEARCH_SNIPPET_MAX_CHARS (default 150) so operators can tune it. Single-asterisk italic is intentionally not stripped to avoid false positives on math/code notation like a * b. Commit: feat(AI-179): env-backed SEARCH_MAX_SOURCES with hard cap; strip markdown and truncate snippets
…down and truncate snippets Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
…ateSnippet Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
…ted to avoid mock breakage Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
roberthunterjr
left a comment
There was a problem hiding this comment.
@copilot A few tweaks here:
Lets use the perplexity client for the search api as seen here, instead of direct API call:
https://docs.perplexity.ai/docs/search/best-practices
Lets ensure the client for Search API also respects the domain filtering defaults or configuration from the .env for filtering.
The snippets revealed as search results are still too long, lets make the snippet truncate after 160 words.
Lastly, lets clear the deliniation between search results. Either lets improve markdown or try something like block kit cards to make the seperation of results and their source and snippets clearer.
…er, word truncation, block kit Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
…lock Kit cards Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Done in commit
|
There was a problem hiding this comment.
Pull request overview
Implements the /fiona search <query> capability in the Fiona Slack app by calling Perplexity’s Search API (non-streaming) and returning 3–5 raw source snippets/links without any synthesized answer. The PR also expands the feedback/telemetry plumbing to tag feedback with response/interaction context (e.g., synthesis vs search; slash vs mention) and updates the command routing so search works via both slash commands and say()-based entry points.
Changes:
- Add a search abstraction (
search-caller) that re-exports the LLM-layer search call and formats results as Slack mrkdwn + Block Kit (with snippet truncation + escaping). - Implement
/fiona searchand say()-based “search” command handling, including rate limiting, immediateack(), formatted responses, andslash_searchinteraction recording. - Extend feedback blocks/handling to encode response type + interaction type in
block_id, and record those fields in Cosmos feedback records; update tests accordingly and add the Perplexity Search SDK dependency.
Reviewed changes
Copilot reviewed 24 out of 25 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| apps/fiona-slack/src/agent/llm-caller.js | Adds Perplexity Search API client + searchForSources() implementation. |
| apps/fiona-slack/src/agent/search-caller.js | New wrapper for search + Slack formatting/escaping/snippet truncation. |
| apps/fiona-slack/src/listeners/commands/fiona.js | Implements /fiona search flow (ack → search → respond), rate limiting, and feedback block tagging. |
| apps/fiona-slack/src/listeners/commands/command-handler.js | Adds say()-based search handler + routes “search” keyword to it; updates help text. |
| apps/fiona-slack/src/listeners/commands/command-dispatch.js | Threads interactionType through keyword routing for feedback tagging. |
| apps/fiona-slack/src/listeners/actions/feedback.js | Parses contextual feedback block ids; attempts to extract search query for search feedback. |
| apps/fiona-slack/src/listeners/views/feedback_block.js | Adds response/interaction-aware feedback block ids + helpers; exports FEEDBACK_RESPONSE_TYPES. |
| apps/fiona-slack/src/listeners/views/feedback_reason.js | Records feedback with response/interaction context; fetches context differently for synthesis vs other response types. |
| apps/fiona-slack/src/listeners/events/app_mention.js | Tags synthesis feedback blocks with interaction type for mentions. |
| apps/fiona-slack/src/listeners/assistant/message.js | Tags synthesis feedback blocks with interaction type for assistant-thread messages. |
| apps/fiona-slack/src/agent/feedback-store.js | Records responseType in Cosmos feedback documents (defaults to synthesis). |
| apps/fiona-slack/src/agent/feedback-response-types.js | New shared enum for feedback response categories. |
| apps/fiona-slack/tests/agent/search-caller.test.js | New test suite for search normalization/formatting/escaping and error paths. |
| apps/fiona-slack/tests/listeners/commands/fiona.test.js | Replaces search stub tests with real search behavior + rate limit coverage. |
| apps/fiona-slack/tests/listeners/commands/command-handler.test.js | Adds tests for say()-based search routing/formatting and help text updates. |
| apps/fiona-slack/tests/listeners/events/app-mention.test.js | Updates mention “search …” expectations from coming-soon to search results. |
| apps/fiona-slack/tests/listeners/assistant/message.test.js | Updates assistant-thread “search …” expectations from coming-soon to search results. |
| apps/fiona-slack/tests/listeners/actions/feedback.test.js | Updates feedback action tests to include contextual block_id and metadata. |
| apps/fiona-slack/tests/listeners/views/feedback-block.test.js | Adds coverage for feedback block id build/parse + custom feedback blocks. |
| apps/fiona-slack/tests/listeners/views/feedback_reason.test.js | Adds search-feedback recording tests and conversation.history mocking. |
| apps/fiona-slack/tests/agent/feedback-store-cosmos.test.js | Adds responseType field coverage + defaulting behavior. |
| apps/fiona-slack/tests/agent/escalation.test.js | Updates llm-caller mock to include searchForSources. |
| apps/fiona-slack/package.json | Adds @perplexity-ai/perplexity_ai dependency. |
| apps/fiona-slack/package-lock.json | Locks @perplexity-ai/perplexity_ai@0.37.0. |
| apps/fiona-slack/.env.sample | Documents optional /fiona search env settings. |
Files not reviewed (1)
- apps/fiona-slack/package-lock.json: Generated file
Comments suppressed due to low confidence (3)
apps/fiona-slack/src/listeners/actions/feedback.js:36
extractSearchQuery()returns the mrkdwn-escaped query (e.g.,&,<,>), which then gets stored asuserMessagefor search feedback. Unescape these entities so feedback analytics/store reflect the user’s original input.
function extractSearchQuery(messageText) {
if (typeof messageText !== 'string') return null;
const match = messageText.match(SEARCH_QUERY_PATTERN);
return match?.[1] ?? null;
}
apps/fiona-slack/src/listeners/actions/feedback.js:95
- To avoid relying on
conversations.history()(which won’t return ephemeral messages), include the rated message text inprivate_metadatawhen opening the feedback modal. This lets the view submission callback recordbotResponseeven for ephemeral/fiona searchresponses.
private_metadata: JSON.stringify({
channelId: channel_id,
messageTs: message_ts,
userId: user_id,
value,
apps/fiona-slack/src/listeners/views/feedback_reason.js:91
- For non-synthesis feedback you fetch the rated message via
conversations.history(). That won’t work for ephemeral messages (e.g.,/fiona searchresponses), sobotResponsewill frequently be null in stored feedback. Prefer passingbody.message.textthrough the modalprivate_metadatainfeedbackActionCallbackand using it as the primary (or fallback)botResponsehere.
if (normalizedResponseType === FEEDBACK_RESPONSE_TYPES.SYNTHESIS) {
({ userMessage, botResponse } = await fetchThreadContext(client, channelId, thread_ts, messageTs));
} else {
botResponse = await fetchMessageText(client, channelId, messageTs);
}
| const { channelId, messageTs, userId, value, thread_ts, responseType, interactionType, searchQuery } = JSON.parse( | ||
| view.private_metadata, | ||
| ); | ||
| const normalizedResponseType = responseType ?? FEEDBACK_RESPONSE_TYPES.SYNTHESIS; |
|
|
||
| import { FEEDBACK_RESPONSE_TYPES, parseFeedbackBlockId } from '../views/feedback_block.js'; | ||
|
|
||
| const SEARCH_QUERY_PATTERN = /^🔍 \*Search results for:\* _"([\s\S]+?)"_/; |
| if (!perplexitySearchClient) return []; | ||
| if (!query || !query.trim()) return []; | ||
|
|
||
| const cappedMaxSources = Math.min(maxSources, SEARCH_ABSOLUTE_MAX); |
| // Collapse newlines and runs of whitespace to a single space | ||
| let cleaned = text | ||
| .replace(/\n+/g, ' ') | ||
| .replace(/\s{2,}/g, ' ') | ||
| .trim(); | ||
| // Strip bold markers (**text**) and heading markers (## …) | ||
| cleaned = cleaned.replace(/\*\*([\s\S]*?)\*\*/g, '$1').replace(/#{1,6}\s+/g, ''); |
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
|
@copilot Please perform another code review that includes the new changes |
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Ran another code review, fixed the follow-up issues it surfaced, and re-checked the updated changes. Latest fix is in |
|
@copilot This is looking nice. I notice that when performing the slash search with an at mention, the results are posted in the channel directly rather than ephemerally. Lets correct this to adhere to the acceptance criteria of the ticket. |
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Addressed in |
|
@copilot The search is working in the side panel and in the Fiona DM, however tagging in a channel with the search keyword does not yield any results, neither ephemeral or posted as a thread directly. |
Co-authored-by: roberthunterjr <8196647+roberthunterjr@users.noreply.github.com>
Fixed in |
/fiona search <query>was a "coming soon" stub. This implements the full skill: calls Perplexity non-streaming, returns 3–5 source snippets with hyperlinks, and deliberately omits any synthesized answer.Core changes
src/agent/llm-caller.js— newsearchForSources(query, options): non-streaming Perplexity call that preferssearch_results(title + snippet) overcitations(URL-only fallback), capped at 5 resultssrc/agent/search-caller.js(new) — abstraction layer re-exportingsearchForSources; also ownsformatSearchResults()(Slack mrkdwn renderer) andescapeMrkdwn()(HTML-entity-encodes&/</>in user input before embedding in message strings)src/listeners/commands/fiona.js—handleSearch()replaces the stub: empty query falls back to help; rate-limit exceeded responds ephemerally; otherwiseack()immediately → search →respond()with formatted sources; recordsslash_searchsrc/listeners/commands/command-handler.js— addshandleSearchViaSay()for @-mention/assistant-panel entry points;routeCommandViaSay()dispatchessearchthere; removesSEARCH_NOT_YET_TEXT; drops "(coming soon)" fromHELP_TEXTsearch lineExample output (Slack mrkdwn)
No results →
🔍 No sources found for _"query"_. Try rephrasing your query.Test surface
tests/agent/search-caller.test.js(new, 38 tests):searchForSourceswith search_results/citations/empty/error paths;formatSearchResults;escapeMrkdwntests/listeners/commands/fiona.test.js: search sub-command replaced with real-behavior tests (empty query → help fallback, query → ack+respond+record, missing fields, rate limiting)tests/listeners/commands/command-handler.test.js: mockssearch-caller, addshandleSearchViaSaytestsapp-mention,message, andescalationtest files updated: addedsearchForSourcesto theirllm-callermocks and updated the "coming soon" search assertions